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ABSTRACT 


In  existing  man-machine  systems,  the  human  operator 
is  typically  presented  a predesigned  display  and  he  must  adapt  his 
control  techniques  in  an  attempt  to  optimize  overall  system  per- 
formance. Ultimate  performance  achieved  may  not  rise  to  the 
designer's  expectation  or  to  the  operator's  capability  because  the 
design  process  does  not  account  for  the  operator's  adaptability. 

In  this  research  program,  ship  control  performance  of  the  Officer 
of  the  Deck  (OOD)  was  observed  and  analyzed  in  a series  of  simu- 
lator experiments  involving  ship  transit  and  obstacle  avoidance. 

Three  types  of  displays  were  included  in  the  analysis.  OOD  control 
rules,  and  measures  and  criteria  which  describe  the  control  technique 
used,  as  well  as  the  criteria  employed  in  controlling  the  ship,  were 
derived  from  the  individual  performance  data.  The  effect  of  display 
type  on  the  measures  and  criteria  which  were  apparently  optimized 
by  the  subjects  was  determined. 
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INTRODUCTION 


1 .0 

The  designer  of  a man-machine  system  typically  performs 
his  design  task  with  knowledge  of  system  objectives,  human  factors 
principles,  and  display  and  control  requirements . However,  it  is  the 
human  operator  who  adapts  his  control  rule  (his  input/output  control 
characteristics)  so  that  the  overall  system  response  satisfies  (to  the 
degree  possible)  his  performance  criteria.  The  performance  that  is 
actually  achieved  will  be  obtained  in  cooperation  with  a system  that  has 
a good  system  design  — and  in  spite  of  a system  having  a poor  design. 

The  above  argument  suggests  that  the  designer  should  have 
available  as  a design  tool  a means  for  estimating  the  operator's  per- 
formance criteria  and  his  control  actions.  The  designer  would  like  to 
know  which  design  features  support  performance  and  which  features 
degrade  performance.  The  need  for  such  knowledge  suggests  several 
research  questions: 

1 . How  can  the  "apparent  Operator's  Measure  and 
Criteria  (OMAC)"  be  determined?  (The  term 
"apparent  OMAC"  is  used  to  mean  those  measures 
and  criteria  that  are  optimized  by  the  operator's 
observed  control  actions.) 

2.  How  can  the  "apparent  OMAC"  be  used  to  predict 
operator  control  actions  in  specific  problem 


situations? 


r 
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3.  Does  the  design  of  the  system  controls  and 
displays  affect  the  "apparent  OMAC"? 

4.  How  can  the  "apparent  OMAC"  be  used  to 
evaluate  alternative  displays? 

Questions  1 and  2 refer  to  processes  that  are  inverse  to 
each  other.  Question  1 asks:  given  the  operator's  control  actions, 

can  we  determine  his  OMAC?  And  Question  2 asks:  given  OMAC, 

can  we  predict  the  operator's  control  action  in  each  problem  situation? 
Optimal  control  theory  provides  a means  to  relate  OMAC  to  the  control 
actions  which  optimize  the  OMAC  although  the  solution  may  not  be 
unique.  For  example,  there  may  be  more  than  one  OMAC  optimized 
by  a given  set  of  control  actions . 

Question  3 refers  to  the  determination  of  the  effect  of 
display  and  control  equipment  on  the  OMAC.  To  understand  the 
significance  of  this  problem  it  is  necessary  to  distinguish  between 
OMAC  and  the  objectives  of  the  control  system  combined  with  any 
instructions  or  standard  operating  rules  that  may  be  presented  to  the 
operator.  The  latter,  the  system  objective  and  instructions , provide 
overall  guidance.  Examples  are  "direct  the  ship  to  the  objective  point 
and  arrive  there  in  one  hour",  and  "remain  in  channel".  In  order  to 
satisfy  these  instructions,  the  operator  will  develop  OMAC  to  evaluate 
the  system  response  and  select  appropriate  actions  on  a moment-to- 
moment  basis.  We  should  note  that  the  OMAC  may  or  may  not  be 
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totally  consistent  with  the  instruction  provided  to  the  operator.  He 
may  make  trade-offs  which  may  lead  to  violations  of  the  operation 
rules . 

An  answer  to  Question  4 would  give  the  designer  some 
way  of  selecting  among  alternative  designs  based  on  the  performance 
expected  from  each. 

The  identification  of  the  OMAC  and  the  development  of 
methods  of  converting  OMAC  to  control  actions  in  each  problem 
situation  provides  a new  way  of  representing  (modeling)  the  human 
operator.  There  are  several  potential  benefits  of  such  a model.  One 
is  that  OMAC  may  be  predictable  over  a wide  range  of  problems  and 
problem  situations  and,  thus,  may  be  used  to  predict  human  response 
in  those  situations.  A human  response  model  using  OMAC  (i.e., 
representing  the  human  operator  by  the  criteria  he  tends  to  optimize) 
contrasts  with  the  input/output  (stimulus/response)  model.  This  latter 
type  of  model  is  extensively  used  to  represent  human  response  but  it 
suffers  from  several  difficulties  among  which  are  - they  are  problem 
specific  and  they  are  generally  limited  to  representing  linear  control 
policies . 

1 . 1 Research  Objectives 

One  objective  of  the  research  was  to  develop  a design 
tool  consisting  of  the  apparent  OMAC  used  by  operators  in  performing 
their  tasks.  This  required  development  of  a method  for  identifying 
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the  apparent  OMAC  from  experimental  data  and  development  of  a 
method  for  applying  an  OMAC  to  predict  operator  control  actions  and 
the  resultant  ship  response. 


A secondary  objective  was  to  develop  a performance 
measurement  for  ship  control  and  with  this  measure  evaluate  the 
effect  of  different  displays. 

The  investigation  reported  here  used  data  collected  during 
a series  of  experiments  in  which  a subject  acting  as  an  Officer-of-the- 
Deck  (OOD)  controlled  a simulated  ship  in  a simulated  environment. 

His  task  was  to  direct  a ship  transit  from  the  initial  point  to  the 
terminal  point  within  a pre-specified  time  intervat  while  avoiding  simu- 
lated contacts  along  the  way.  The  experiment,  using  equipment  known 
as  the  Surface  Ship  Bridge  Console  System  (Gawitt/Beary),  was  run  by 
personnel  of  NSRDC,  Annapolis,  Maryland,  for  purposes  other  than  this 
research  program.  The  data  from  that  experiment  was  used  in  the 
research  reported  here . 

1 .2  Summary  of  Results 

Data  from  the  experiment  was  analyzed  and  statistically 
significant  differences  in  performance  using  different  displays  were 


found.  The  differences  are  associated  with  the  differences  in  the 
ability  of  the  subject,  using  a particular  display,  to  maneuver  early 
in  order  to  avoid  passing  close  to  a contact. 
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A method  for  computing  the  apparent  OMAC  and  using 

- 

OMAC  to  predict  operator  control  actions  was  developed.  The  require- 
ment was  demonstrated  for  a purview  function  in  the  OMAC  which 
represents  a range  limit  of  the  contacts  actively  considered  by  the  OOD. 

Evidence  to  support  the  hypothesis  that  display  type  affects  OMAC  is 
presented . 
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2.0  METHOD 

The  objectives  of  the  research  were  accomplished  with 
four  analyses.  Analysis  I was  to  analyze  performance  data  from  the 
experiment  using  summary  measures.  Its  purpose  was  to  determine 
if  performance  is  actually  different  with  different  displays  and  to 
provide  results  for  Analysis  II. 

Analysis  II  was  also  a performance  study  but  used  measures 
that  detect  control  actions  that  lead  to  critical  conditions  (near  collisions). 
In  addition.  Analysis  II  provides  the  performance  measure  required  to 
compare  synthesized  (optimal)  ship  responses  to  the  subject  ship  res- 
ponses obtained  from  the  experiment.  This  comparison  is  required  to 
identify  apparent  OMAC  from  a set  of  candidate  OMAC. 

Analysis  III  was  an  initial  test  of  OMAC  where  optimal  ship 
responses  were  developed  in  a one-contact  problem.  The  purpose  of 
the  analysis  is  to  establish  a set  of  factors  to  be  included  in  OMAC  for 
test  in  Analysis  IV. 

Once  a set  of  OMAC  factors  were  established,  the  apparent 
OMAC  was  determined  in  Analysis  IV.  Composite  performance  data 
representing  performance  of  subjects  working  with  a particular  display 
are  compared  to  performance  data  from  the  set  of  OMAC.  From  that 
comparison,  the  OMAC  apparently  used  by  the  operators  with  each  type 
of  display  are  identified. 
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2.1 


Description  of  the  Manual  Control  Task 


The  subject  acting  as  an  Officer  of  the  Deck  (OOD)  commands 
the  ship  course  and  speed  required  to  transit  the  simulated  ship  from  the 
initial  position  to  the  final  position  while  avoiding  each  simulated  contact 
(other  ship)  encountered.  The  distance  between  the  initial  and  final 
positions  is  approximately  30  miles  and  the  time  allotted  for  the  transit 
is  90  minutes . 

The  subject  sits  in  front  of  a simulator  console  which  has 
three  computer  driven  CRT  displays  and  a number  of  control  switches. 

The  CRT  display  directly  in  front  of  the  subject  is  the  main  display 
and  provides  visual  information  about  the  location  of  (own)  ship  and 
contacts.  The  configuration  of  that  display  varies  with  the  particular 
display  function  simulated,  and  thus,  is  an  experimental  variable.  The 
CRT  display  on  the  subject’s  left  provides  a list  of  contact  information 
such  as  location,  speed,  heading,  and  closest  point  of  approach  (CPA). 

This  display  is  termed  "LIST".  The  display  on  the  subject's  right 
provides  data  on  ship  course  and  speed.  It  can  also  provide  data  on  "trial" 
course  and  speed  which  the  subject  may  input  for  evaluating  decision  command 
changes.  There  are  a number  of  control  switches  which  allow  the  subject  to 
select  display  functions,  to  input  trial  course  and  speed,  and  to  direct  course 
and  speed  change  commands  to  an  individual  representing  the  Helmsman. 

The  task  of  the  subject  OOD's  is  to  visualize  or  calculate 
the  possible  courses  from  present  ship  position  to  the  objective  point. 
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Then,  by  taking  into  account  the  contact  positions  and  velocities  and  by 


utilizing  the  various  display  functions  available,  he  is  to  direct  the  ship 
to  the  objective  position.  The  instructions  read  to  the  subject  prior 
to  each  trial  are  "the  objectives  during  the  run  are  to  be  at  the  rendez- 
vous at  the  end  of  the  90  minute  period,  to  pass  clear  of  all  contacts 
(if  possible  by  more  than  4,000  yards),  to  obey  the  Rules  of  the  Road, 
and  to  observe  economy  of  operation". 

Performance  with  three  types  of  displays  is  of  interest. 

The  three  types  are: 

1 . "OLD  system" 

2.  "PACS  system" 

3.  "RVV  system" 

The  OLD  system  is  essentially  a conventional  radar  display  where  the 
center  of  the  screen  is  the  position  of  own  ship  and  the  relative  position 
of  contacts  are  shown  as  blips. 

PACS,  a new  display,  provides  a number  of  different  display 
functions  which  are  selectable  by  the  subject.  PACS  (probable  area  of 
collision)  is  the  name  of  the  function  and  the  name  of  the  display  system 
providing  that  function.  In  PACS,  LIST  is  available  on  the  left-hand 
CRT.  Also,  the  subject  can  select  a new  display  function  PACS  or  true  velocity 
vector  modes  on  the  center  CRT.  With  the  PACS  mode,  the  display  provides 
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the  locus  of  collision  points  for  each  contact  for  all  own  ship  courses. 
Since  this  requires  projection  of  future  own  ship  and  contact  positions. 


the  display  aids  the  subject  in  selecting  a new  course.  With  the  true 
velocity  vector  mode,  velocity  vectors  are  superimposed  on  the  own 
ship  and  the  contact  "blips"  to  show  the  predicted  position  of  all  the  ships 
at  a future  time  selected  by  the  operator. 

Finally,  another  new  display  termed  "RVV  system"  includes 
LIST  plus  relative  velocity  vectors.  These  are  vectors  superimposed 
on  contact  blips  that  indicate  the  relative  position  of  each  contact  with 
respect  to  own  ship  at  a future  time  selected  by  the  operator. 


2.2  Method  of  Analysis 

2.2.1  Analysis  I:  Summary  Performance  Measurement 


Ten  summary  performance  measures  were  developed  and 
applied  to  each  subject  run.  Since  some  of  the  measures  may  be 
related,  a correlation/factor  analysis  of  all  measures  for  all  subject 
runs  (a  total  of  40  runs)  was  performed.  This  led  to  selection  of 
performance  measures  for  the  performance  analysis. 

The  selected  measures  were  then  analyzed  with  a two  factor 
ANOVA  with  a repeated  measure  on  one  factor.  This  permitted  control 
of  the  differences  among  subjects  as  individual  subject  performance 
measures  are  related  to  his  average  performance.  Only  eight  subjects 
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(four  for  head-up1  and  four  for  north-up1)  have  run  under  all  display 
systems  (OLD-PACS-RW).  Thus  not  all  subject  data  are  used  to 
evaluate  the  summary  measures.  The  model  for  the  ANOVA  is  given 

in  Table  1 . 


The  summary  measures  used  are  listed  below  and  defined 
mathematically  in  Table  2. 

1 . Summation  of  the  absolute  value  of  course  changes . 

2.  Summation  of  the  absolute  value  of  speed  changes. 

3.  Summation  of  samples*  of  difference  from  reference  speed. 

4.  Summation  of  samples*  of  deviation  from  straight  course 
to  objective . 

5.  Summation  of  samples*  of  CPA  (Closest  Point  pf 
Approach)  violations  where  own  ship  is  privileged. 

6.  Summation  of  samples*  of  CPA  violations  where  own 
ship  is  burdened. 

7.  Summation  of  samples*  of  the  inverse  of  time  to  CPA 

given  a projected  CPA  violation. 

8.  Total  travel  distance. 

9.  Distance  from  objective  at  end  of  allotted  time. 

10.  Number  of  decisions. 

*Samples  taken  each  minute 

1 "Head-up"  is  a display  mode  where  the  top  of  display  corresponds  to 
ship's  course.  "North-up"  is  a display  mode  where  the  top  of  display 
corresponds  to  north. 
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where  A*,  is  the  number  of  degrees  of 

i 

th  ! I 

change  of  the  i course  change,  ; is  the 


absolute  value  operation,  is  the  number 


of  course  changes 


ASi  * 


where  AS.  is  the  i speed  change, 
N2  is  the  number  of  speed  changes. 


'k  Rk  I » 


where  S,  is  the  own  ship  speed  on 
k 

th 

the  k time  interval,  S is  the 

Kk 

th 

reference  speed  on  the  k time 


interval,  N is  the  number  of  time 

O 

intervals. 


Ji  - 4 

i rRi 


S.  * AT, 
i 


where  is  own  ship  actual  course, 

^ is  course  from  own  ship  present 

position  to  the  objective  point,  S.  is 

speed,  AT  is  duration  of  time  interval 

between  samples  of  process,  i refers 
.th 

to  the  i course  change. 


TABLE  2 SUMMARY  PERFORMANCE  MEASURE 
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5.  F_  = 


6.  F_  = 


> - £ 


NL  N„ 
3 4 


k=1  i=1 


(4000  - CPA,  .)  M 

ki  ki  * 


where  CPA,  . is  the  CPA  of  own 
ki 

ship  and  contact  i on  the  k time 
interval  M equals  1 if  own  ship 

K l 

is  privileged  and  if  the  CPA  is 


less  than  4000  yards.  M,  . is 

ki 


zero  otherwise.  N is  the  number 

4 


of  contacts. 


(4000  - CPA,  .)  N,  . , 
kr  ki 


TCPA  . * 
ki 


where  N.  . equals  1 is  own  ship 
ki 

is  not  privileged  and  if  the  CPA 


is  less  than  4000  yards.  N,  . is 

ki 


zero  otherwise. 


where  TCPA,  , is  the  time  to  CPA 
ki 

th 

of  own  ship  and  contact  i on  the  k 

time  interval.  The  smallest  value  of 

TCPA  permitted  is  1,  i.e.,  if  the  time 

to  CPA  is  less  than  one,  TCPA  is  set 

to  one.  P,  . equals  1 if  CPA,  . is  less 
ki  ki 

than  4,000  yards.  P equals  zero  otherwise, 

KI 


TABLE  2 SUMMARY  PERFORMANCE  MEASURE  (CONTD) 
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K 


k=1 


where  K is  a constant  which  converts 
speed  over  a one  minute  interval  to 
distance. 


9.  F8  = D (T), 


D is  distance  to  objective.  T is 
time  at  end  of  experiment. 


equals  1 if  speed  or  course 
has  changed  during  the  interval 
equals  zero  otherwise. 


TABLE  2 SUMMARY  PERFORMANCE  MEASURE  (CONTD) 
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The  first  and  second  measures  are  sums  of  the  absolute 


values  of  course  and  speed  change  values.  These  measures  combine  the 
number  of  OOD  commands  with  amount  of  change  in  ship  course  and 
speed.  The  third  measure  is  the  displacement  (at  the  end  of  the  interval)  of 
the  ship  from  a path  from  present  ship  position  to  the  objective  point.  The 
reference  course  and  ship  course  required  for  the  measure  are  illustrated  in 
Figure  1 . The  measure  is  intended  to  measure  the  amount  of  deviation  from  the 
straight  line  to  the  objective.  The  fourth  measure  is  the  sum  of  the 
absolute  / value  of  the  deviation  from  a reference  speed.  The  reference 
speed  is  the  speed  required  to  reach  the  objective  within  the  time 
remaining,  times  a correction  factor.  The  correction  accounts  for  the 


delay  caused  by  contact  avoidance  maneuvering.  Measure  5 is  a sum- 


mation of  the  amount  of  CPA  violation  (distance  to  contact  less  than 
4,000  yards)  where  the  own  ship  is  in  a privileged  orientation  with 
respect  to  the  contact.  A privileged  orientation  as  defined  by  the  rules 
of  the  road  (Zadig,  1972),  provides  that  the  contact  is  responsible  for 
making  contact  avoidance  maneuvers.  Measure  6 is  the  same  as  5 but 
is  computed  in  those  situations  for  which  own  ship  is  burdened  (i.e., 
responsible  for  contact  avoidance  maneuvering)  and  all  other  situations 
that  do  not  fall  in  the  privileged  category.  Measure  7 is  the  reciprocal 
of  time  to  the  closest-point-of-approach  (CPA)  for  all  contacts  where  a 
contact  violation  is  projected.  The  measure  value  increases  as  time- 


to-CPA  becomes  smaller.  Measure  8 is  the  total  distance  travelled 
by  the  subject  from  start  to  end  of  allotted  time.  Distance  from  own 
ship  position  to  the  objective  point  at  the  end  of  allotted  time  is  given 
by  Measure  9.  Measure  10  is  the  number  of  speed  and  course  change 
decisions  made  by  the  subject. 

2.2.2  Analysis  II:  Contact  Avoidance  Maneuvering 

2.2.2. 1 CPA/TCPA  Matrix  According  to  the  instructions 
provided  to  the  operator,  he  is  to  direct  the  ship  to  the  objective  point 
within  the  prescribed  time  while  avoiding  contacts  by  at  least  4,000 
yards  if  possible.  Analysis  of  how  he  accomplishes  the  task  requires 
a method  of  dividing  the  problem  situation  into  categories  so  that  pei — 
formance  in  each  category  can  be  examined.  The  selected  method 
uses  a CPA/TCPA  matrix  as  shown  in  Figure  2.  Cells  in  the  matrix 
correspond  to  intervals  of  CPA  and  TCPA.  At  each  time  interval,  the 
state  of  the  ship  (position,  speed,  course)  and  state  of  each  contact 
specify  a cell  condition.  For  N contacts  there  are  N cell  conditions. 
For  example.  Cell  1 corresponds  to  a projected  CPA  of  from  0 to  1.5 
miles  which  is  projected  to  occur  within  0 to  5 minutes.  Cell  21  rep- 
resents the  situation  where  the  CPA  has  already  occurred,  i.e.,  range 
to  the  contact  is  increasing.  Projected  CPA  violations  (passing  within 
4,000  yards  (2  miles)  ) are  identified  by  entries  in  Cells  1 through  12 
and  Cell  19. 

^CPA  is  closest  point  of  approach  and  TCPA  is  time  to  CPA. 
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As  an  aid  to  understanding  how  the  entries  in  the  CPA/ 

TCPA  matrix  change  as  a function  of  ship  and  contact  position  changes, 
consider  the  following  typical  situations.  If  the  ship  speed  and  course 
remain  constant  over  several  time  intervals,  cell  conditions  change  to 
decrease  TCPA.  For  example,  an  entry  in  Cell  6 would  "transfer" 
to  Cell  5 because  of  the  elapsed  time. 

If  ship  course  or  speed  is  changed  by  the  operator  as  a 
contact  avoidance  or  transiting  maneuver,  other  cell  transitions  occur. 

As  an  example,  an  entry  in  Cell  6 identifies  a condition  where,  unless 
ship  (or  contact)  course  or  speed  is  modified,  the  ships  will  pass  within 
4,000  yards  in  30  minutes.  If  no  action  is  taken,  the  entry  in  Cell  6 
will  transfer  to  Cell  5 because  of  the  elapsed  time.  In  order  to  avoid 
a contact  violation  (passing  within  4,000  yards),  it  is  necessary  that  the 
OOD  command  a change  of  ship  speed  or  course  such  that  the  entries 
are  transferred  to  a cell  without  a projected  violation. 

2. 2. 2. 2 Transition  Matrix  Cell  transition  patterns 

characterize  the  OOD's  contact  avoidance  and  transit  maneuvering.  Like- 
wise, transition  patterns  representing  a group  of  OOD's  (such  as  those 
using  a particular  display)  can  be  used  to  characterize  the  group  maneuver 
patterns. 

Cell  transitions  are  represented  by  transition  probabilities 
since  cell  sequences  are  not  fixed  - even  those  representing  performance 
of  an  individual  performer.  The  transition  pattern  from  a given  cell. 
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say  Cell  i,  is  recorded  in  the  associated  row  of  a 21  by  21  transition 
matrix.  Probability  ^P_^  is  the  Probabl^ty  of  transferring  to 
Cell  j from  Cell  i on  each  time  interval.  In  the  experiment,  data  was 
sampled  every  minute  so  that  the  probabilities  are  associated  with  the 
indicated  transfer  in  each  one  minute  interval.  Entries  in  the  diagonal 
elements  of  the  matrix,  where  i equals  j,  have  a special  meaning. 
Probability  values  P are  related  to  the  mean  number  of  intervals  for 
which  an  entry  in  Cell  i will  remain  in  Cell  i.  P is  computed  accord- 


ing to  the  formula: 


n 


1 - 1 
I 


where  I is  the  mean  number  of  intervals  an  entry  is  in  Cell  before 

ii 

transition  to  another  cell.  Calculation  of  the  probabilities  off  the 
diagonal  from  the  experiment  data  uses  the  formula: 

N. . 

p • = — (1  ~ P • ),  i ¥■  j 

ii  M.  u J 

i 

where  N_  is  the  number  of  transitions  from  Cell  i to  Cell  j and  M.  is 

the  total  number  of  transitions  from  Cell  i.  The  term  Cl  - P ) is 

ii 

the  probability  that  a transfer  to  another  cell  will  occur  in  each  time 
interval . 

Although  the  above  description  of  the  CPA/TCPA  matrix, 
the  transition  matrix,  and  the  method  of  calculating  transition  proba- 
bilities is  presented  as  though  cell  entry  changes  occur  for  only  one 
contract  at  a time,  there  are  in  fact  multiple  simultaneous  changes. 
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Each  course  or  speed  change  commanded  by  the  OOD  can  cause  multiple 
simultaneous  changes  in  the  cell  entries.  Transition  probabilities  are 
developed  from  transition  events  that  occur  both  simultaneously  and 
throughout  the  experiment.  When  a transition  matrix  is  developed  to 
represent  performance  of  a group  of  subjects,  data  used  is  obtained 
from  all  subject  runs.  In  spite  of  the  fact  that  the  transition  matrix 
probabilities  represent  both  simultaneous  and  sequential  occurrences 
and  multiple  subject  runs,  it  is  often  convenient  to  visualize  the  tran- 
sition matrix  as  describing  the  relationship  between  own  ship  and  one 
contact  - in  the  presence  of  many  other  contacts.  Thus,  it  is  possible 
to  trace  a likely  path  of  cells  from  the  point  of  contact  activation  until 
the  ship  reaches  the  objective  point.  Different  contact  initial  conditions 
(conditions  at  time  of  contact  activation)  may  lead  to  different  cell 
sequences . 

2. 2. 2. 3 Comparison  of  Ship  Response  Patterns 
Considerable  analysis  of  ship  maneuvering  patterns  can  be  accomplished 
by  inspection  of  the  transition  matrix  probabilities.  For  example,  an 
entry  in  Cell  6 corresponds  to  a projected  CPA  violation  in  25  - 30 
minutes.  Transitions  to  Cells  5,  4,  3,  2,  1 will  occur  if  no  OOD 
control  actions  exist  to  cause  a transfer  to  cells  where  CPA  violations 
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are  not  projected.  Also,  as  time  to  CPA  decreases  in  cells  where  a 
CPA  violation  is  projected,  the  less  desirable  the  situation  represented 
by  the  cell.  Thus  examination  of  transition  probabilities  provides  a 
way  of  evaluating  performance  and  comparing  performance  with  different 
experimental  treatments. 

One  difficulty  in  the  quantitative  evaluation  of  performance, 
using  transition  probabilities  alone,  is  that  transition  from  a given  cell  is 
not  weighted  by  the  probability  the  cell  is  "used".  For  example,  with 
one  treatment  ( one  display  type),  transition  from  Cell  3 may  not  be  as 
good  (a  high  probability  of  transfer  to  a cell  that  does  not  project  a CPA 
violation)  as  that  of  another  treatment.  But  the  probability  that  an  entry 
will  be  in  Cell  3 may  be  so  small  that  transitions  from  Cell  3 may  be  of 
little  consequence.  A method  for  solving  this  problem  is  described  in 
the  next  paragraph  after  introduction  of  a required  concept. 

Once  a contact  is  passed,  where  distance  from  contact  is 
increasing,  there  is  little  likelihood  that  the  ship  will  maneuver  so  as  to 
approach  that  contact  again.  Assuming  that  condition  is  true.  Cell  21 
can  be  treated  as  an  absorbing  cell,  i.e.,  the  probability  of  leaving  the 
cell  after  the  cell  is  entered  is  zero.  With  that  assumption,  the  tran- 
sition matrix  can  be  analyzed  as  an  absorbing  Markov  Process  (Kemeny 
& Snell,  1960).  This  means  that  the  process  is  considered  as  starting 
in  any  cell  (but  especially  Cells  19  and  20)  and  transferring  to  a series 
of  other  cells  ending  in  Cell  21  . An  entry  may  return  to  a given  cell 
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in  that  process  but  once  entering  Cell  21,  it  remains  there. 


The 


I 


:l 

ii 


i 
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analysis  of  interest  is  the  calculation  of  the  mean  number  of  times 
each  cell  will  be  entered  before  entering  the  absorbing  state  given  the 
starting  cell.  This  provides  the  desired  measure  of  cell  usage. 
Calculation  of  the  measure  required  formulation  of  a matrix  Q which 
is  identical  to  the  transition  matrix  P except  that  row  21  and  Column  21 
(corresponding  to  the  absorbing  cell)  have  been  removed.  The  desired 
measure  is  given  by  matrix  R (a  20  X 20  matrix): 

R = (I  - Q)_1 

where  I is  the  identity  matrix,  "-1"  represents  the 

matrix  inverse  operation,  and  elements  r of  R are 

ij 

the  mean  number  of  times  the  process  enters  Cell  j 

given  the  process  starts  in  Cell  i. 

The  next  and  final  step  in  the  comparison  of  ship  response 
patterns  is  to  compare  subject  controlled  ship  response  for  each  display 
to  each  of  the  optimal  ship  responses.  MT(I)  is  used  to  denote  the 
mean  number  of  times  Cell  I is  entered  before  the  process  enters  the 
absorbing  cell  (21).  Comparison  is  made  with  the  sum  of  the  squared 
differences  of  the  cell  mean  entries  (MT(I)).  The  comparison  is  made 
of  the  MT  in  Cells  1-5  and  7-11,  deemed  the  critical  region  cells 
since  they  represent  a situation  where  a CPA  violation  (less  than  2 mile 
CPA)  is  projected.  The  Wilcoxon  matched-pair  signed-ranks  test  was 
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used  for  the  comparison.  Subjects  performing  in  each  set  of  two 
displays  provided  the  matched  pairs.  Performances  with  the  three 
displays  were  compared  two  at  a time. 

2.2.3  Synthesis  of  the  Optimal  Ship  Response 

2. 2. 3.1  Coordinate  System  Dynamic  programming 
(Bellman,  1962,  Elgerd,  1967)  is  a method  for  computing  optimal  ship 
responses  which  utilizes  a discrete  analysis  technique.  The  area 
between  the  ship  starting  position  and  objective  position  is  approximated 
by  a grid  as  illustrated  in  Figure  3 . A ship  path  is  approximated  by 
the  straight  lines  connecting  the  grid  intersections  as  illustrated  in  the 
figure.  For  example,  a ship's  path  is  represented  by  identifying  the 
sequence  of  grid  intersections  (identified  as  the  X,Y  coordinates)  as  a 
function  of  time.  In  this  way,  a smooth  path  is  represented  approximately 
by  a path  of  straight  line  segments  as  shown  in  the  figure. 

With  this  grid,  the  problem  of  finding  a continuous  optimal 
path  is  converted  to  one  of  finding  the  locus  of  points  on  the  grid  that, 
whe  connected  by  straight  lines,  provide  the  optimum  grid  path  from 
the  starting  position  (SP)  to  the  objective  position  (OP).  This  path  is 
taken  as  an  approximation  to  the  continuous  optimal  path. 

The  grid  structure  selected  has  45  increments  along  the 
basic  course,  i.e.,  the  x axes,  the  line  from  the  SP  to  the  OP.  Six 
levels  of  deviation  from  the  base  course,  three  per  side,  are  provided 
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as  shown  in  Figure  3.  Scaling  of  one  mile  per  interval  in  the  V 
direction  (Y  is  measured  perpendicular  to  the  X direction)  allows  for 


a deviation  of  + 3 miles  from  the  base  course. 

The  ship's  speed  is  also  approximated  by  seven  levels  as 


follows: 


Level  Number 


Speed  (knots) 


Contact  positions  are  superimposed  on  the  grid  as  a continuous  factor 
of  time  so  that  range  to  contact,  projections  of  CPA,  TCPA  can  be 


computed. 


As  an  aid  to  understanding  the  calculation  by  which  the 


optimum  path  is  determined  (to  be  described  in  the  next  section),  consider 
the  problem  of  specifying  a ship  path  (not  necessarily  the  optimum  path) 
from  the  start  to  the  destination.  Referring  to  Figure  4,  consider  that 
the  ship  is  at  the  starting  position.  By  examining  the  position,  course 
and  speed  of  each  observable  (activated)  contact,  a feasible  path  to  the 
destination  must  be  envisioned.  The  path  is  to  be  designated  by 
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specifying  the  grid  intersections  and  ship  speed  along  each  path  segment 
connecting  the  grid  intersections.  The  path  is  to  be  selected  so  that  the 
minimum  range  to  each  contact  is  not  less  than  4,000  yards  and  the 
destination  is  reached  within  the  allowed  90  minutes.  The  first  path 
interval  consists  of  a segment  from  the  initial  position  where  X equals  0 , 
to  one  of  the  grid  intersections  where  X equals  1 . The  particular  intei — 
section  selected  at  that  interval  is  identified  by  a value  of  Y (in  this  case 
2).  Along  that  segment,  a speed  level  number  is  selected  as  indicated  in 
the  left  portion  of  the  figure.  Obviously,  this  procedure  is  repeated  for 
increasing  values  of  X until  the  objective  position  is  reached. 

2. 2. 3. 2 Method  of  Calculating  Optimum  Ship  Path  The 

problem  of  specifying  the  optimum  path  is  similar  to  the  process  des- 
cribed above  except  that  the  path  selected  must  optimize  the  criteria  of 
interest.  Two  types  of  problems  exist  in  specifying  the  optimum  path. 
One  is  the  large  number  of  possible  path  and  speed  combinations  prevent 
direct  evaluation  of  each  possible  combination.  Even  though  the  number 
of  levels  of  deviation  from  the  base  line  (the  values  of  Y)  and  the  number 
of  speed  levels  are  small,  the  number  of  possible  combinations  of  paths 
and  speed  is  large.  Consider  that  on  the  first  interval,  where  X equals 
1 , there  are  seven  Y deviation  levels  and  seven  speed  levels  - a total 
of  49  combinations.  At  the  next  interval,  where  X equals  3,  there  are 
another  49  combinations  of  Y deviation  and  speed.  Each  combination  of 
the  first  interval  could  be  associated  with  a combination  of  the  second 
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interval  to  form  a path,  and  thus  there  are  49  times  49  possible  com- 
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binations  for  the  first  two  intervals.  Extension  of  this  reasoning  for 
subsequent  intervals  reveals  that  for  the  45  intervals,  there  are  49  to 
the  45th  power  possible  path  and  speed  combinations  to  consider.  This 
large  number  of  combinations  prevents  direct  calculation  of  the  value  of 
each  combination  in  order  to  determine  which  is  best.  This  problem 
can  be  solved  using  dynamic  programming  which  capitalizes  on  the  fact 
that  each  portion  of  an  optimal  path  is  also  optimal.  Dynamic  pro- 
gramming provides  a systematic  way  to  develop  candidate  optimal  paths 
step  by  step  along  each  interval  in  the  X axis  until  the  destination  is 
reached  where  the  true  optimal  path  is  identified. 

The  other  problem  referred  to  is  that  contacts  are  activated 
at  various  times  throughout  the  problem  with  some  activated  initially  and 
others  activated  as  the  ship  progresses  toward  the  destination.  A path 
that  is  optimal  with  one  set  of  contacts  may  not  be  optimal  when  an 
additional  contact  is  observed.  Thus,  when  a new  contact  is  activated, 
the  present  position  of  the  ship  is  noted  and  a new  optimal  path  is 
determined  using  present  ship  position  as  a starting  point.  The  cal- 
culation then  proceeds  as  before  but  now  includes  the  new  contact. 

2.2.4  Analysis  III  and  IV:  Method  of  Determining  the 

Apparent  OMAC 

Optimal  control  theory  is  the  connecting  link  between  criteria 
to  be  optimized  and  optimal  control  actions  (which  are  applied  to  the  ship 
to  prodjce  optimal  ship  paths).  Given  the  criteria,  control  objectives 
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(transit  to  destination),  and  limitations  on  control  actions,  optimal 
control  theory  is  used  to  synthesize  the  optimal  control  actions. 

Calculation  of  the  optimal  control  actions  from  criteria  is 
a synthesis  process.  The  inverse  process,  i.e.,  calculation  of  the 
apparent  criteria  (the  criteria  which  is  optimized  by  the  observed  control 
actions),  is  not  a synthesis  process  since  there  is  no  formal  way  of 
solving  for  such  criteria.  However,  the  problem  can  be  solved  by 
repeated  application  of  the  synthesis  procedure  in  which  a set  of 
candidate  criteria  is  selected  and  using  each  criteria,  the  associated 
optimal  control  actions  and  therefore  associated  optimal  ship  response, 
are  synthesized.  The  set  of  optimal  ship  responses  generated  is 
compared  to  the'  ship  responses  observed  in  each  performance  grouping 
of  the  experimental  data.  A performance  group  represents  the  set  of 
experimental  data  for  all  subjects  using  a particular  display.  The  mean 
number  of  entries  in  each  CPA/TCPA  matrix  cell,  described  previously, 
is  used  for  the  comparison.  That  optimal  ship  response  judged  to  be 
closest  to  the  ship  response  of  each  performance  group  identifies,  by 
deductive  reasoning,  the  criteria  to  be  associated  with  the  performance 
group. 

This  method  of  identifying  the  apparent  OMAC  is  illustrated 
in  Figure  5.  It  must  be  noted  that  the  criteria  associated  with  each 
performance  group  is  only  the  most  representative  of  the  set  of  criteria 
initially  selected  and  it  should  also  be  noted  that  the  criteria  may  not  be 
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FIGURE  5 METHOD  OF  IDENTIFYING  THE  APPARENT  CRITERIA  USED  BY 


unique.  Criteria  so  identified  can  be  considered  as  representative  of 
the  actual  criteria  (and  therefore  useful)  only  when  it  is  predictive,  i.e., 
can  accurately  predict  operator's  control  actions  in  problem  situations. 

Synthesis  of  optimal  control  actions  was  accomplished  with 
two  problem  situations.  One  employed  a single  contact  crossing  in 
front  of  own  ship  and  the  other  used  all  25  contacts  in  accordance  with 
the  experiment  design  described  previously.  Evaluation  of  candidate 
OMAC  functions  and  parameter  values  was  first  accomplished  with  the 
simpler  one-contact  problem.  Based  on  the  results  of  the  one-contact 
problem  (Analysis  III),  candidate  OMAC  were  selected  for  test  with  the 
25-contact  problem  (Analysis  IV). 

Contact  and  own  ship  positions  for  the  one-contact  problem 
are  shown  in  Figure  6.  Contact  initial  positions  and  courses  for  the 
25-contact  problem  are  shown  in  Figure  7.  Table  3 provides  the  times 
of  activation  (time  contacts  are  first  presented  to  the  operator).  These 
are  the  contacts  that  must  be  avoided  by  the  subject  OOD  as  he  directs 
the  ship  to  the  objective  point. 

2.2.5  Candidate  OMAC 

A set  of  candidate  OMAC's  were  developed  based  on  the 
instructions  and  operation  rules  given  to  the  subject,  and  the  Maritime 
International  Rules  of  the  Road.  Factors  that  could  be  included  in 
candidate  OMAC  are: 
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Time  Activated 

Contact  Number  (Hours  After  Start  of  Problem) 


1 

O 

O 

2 

0.0 

3 

0.0 

4 

0.0 

5 

0.0 

6 

0.0 

7 

0.0 

8 

0.0 

9 

0.01 

10 

0.08 

1 1 

0.35 

12 

0.40 

13 

0.50 

14 

0.55 

15 

0.90 

16 

1 .00 

17 

1 .01 

18 

1 .08 

19 

1 .18 

20 

1 .20 

21 

1 .26 

22 

1 .35 

23 

1 .41 

24 

1 .46 

25 

1 .50 

TABLE  3 CONTACT  ACTIVATION  TIME 


I 

•! 
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1 . Accumulated  time 

2.  Distance  from  destination 

3.  CPA 

4.  Economy  of  operation  (fuel  usage) 

5.  Rules  of  the  road 

6.  TCPA 

7.  Number  of  decisions 

8.  Total  travel  distance 

The  approach  used  was  to  select  factors  for  an  OMAC,  to 
determine  the  OMAC  coefficients  which  best  predict  experiment  perfor- 
mance data  and  to  evaluate  the  predictive  capability  of  that  OMAC.  One 
function  selected  is: 

C = AxT+Bx  CPA  + C x VtCPA 
where  A,  B,  C are  weighting  coefficients,  T is 
accumulated  time  since  start  of  run,  CPA  is 
closest  point  of  approach,  and  TCPA  is  time  to 
CPA. 


A limit  in  the  weighting  coefficients  was  used  as  follows: 

A + B + C = 1,  A,  B,  C > 0, 

thus  only  two  weightings  can  be  selected  freely  and  those  must  be  less 
than  unity. 
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An  alternate  and  simpler  OMAC  function  was  formed  as: 

C2  = A x T + x CPA/TCPA 
where  A + B = 1,  A,  B > 0 

The  second  OMAC  (C^)  is  a two-factor  system  which  was 
eventually  selected  to  simplify  the  initial  investigations.  Both  OMAC 
constructions  ignore  some  factors  listed  above  which  may  be  important. 
However,  it  was  also  important  to  simplify  the  problem  for  the  "first 
cut" . 
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RESULTS 
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3.1  Analysis  I Results:  Summary  Measure  Results 

Correlation  of  the  values  of  the  ten  summary  measures 

over  all  subject  runs  reveals  that  strong  relationships  exist  between 
several  measures.  Measure  pairs  found  to  be  strongly  related  are: 

1 •* — ►lO 

3-« — ► 8 (Refer  to  page  2-5  for  description  of 

summary  measures) 

4m *.9 

6 m — -*.7 

Analysis  of  Variance  (ANOVA)  on  the  summary  measures 
showed  no  significant  differences  due  to  display  orientation  (head-up  or 
north-up).  Also,  there  are  no  significant  differences  in  performance 
due  to  display  type  at  the  a = 0.05  level.  There  is  significance  of 
performance  due  to  display  type  with  measures  7 and  8 (the  inverse  of 
time-to-CPA  and  total  travel)  at  the  «=  0.1  level. 

3.2  Analysis  11  Results:  Mean  Number  of  Entries  in 

Critical  Region 

The  Wilcoxon  matched-pairs  signed-ranks  test  was  applied 
to  compare  performance  as  follows: 

1 . PACS  vs . RV  V 

2.  PACS  vs.  OLD 

3.  OLD  vs.  RV V 


Results  showed  that  in  comparing  PACS  vs.  RV.V,  PACS  provided 

significantly  fewer  entries  in  the  critical  region  (significant  at  a = .01 

level).  Comparing  PACS  vs.  OLD  showed  PACS  superior  but  significantly 

superior  only  at  the  a = .10  level.  There  was  no  significant  difference 

between  performance  with  OLD  and  RVV.  Data  analysis  Tables  4,  5 

and  6 provide  the  test  results.  Note  that  in  the  PACS  vs.  OLD  analysis, 

only  one  subject  out  of  the  eleven  subjects  performed  better  with  OLD. 

3.3  Analysis  III  Results:  Synthesis  of  Optimal  Controls  for 

One-Contact  Problem 

The  purpose  of  Analysis  III  was  to  evaluate  candidate  OMAC 
functions  so  that  a final  selection  could  be  made  for  the  25  contact 
problem  (Analysis  IV).  While  this  purpose  was  accomplished,  and  the 
results  are  provided  at  the  end  of  this  section,  the  need  to  include  a 
purview  constraint  in  the  optimization  process  was  discovered.  This  is 
an  important  result  because  it  shows  that  modeling  of  subject  performance 
cannot  be  accomplished  without  establishing  a purview  limit.  With  the 
purview  limit,  the  synthesis  considers  all  contacts  within  purview  (range) 
but  ignores  all  others.  However,  the  purview  limit  introduces  an 
additional  parameter  (range  of  purview).  For  this  reason,  the  set  ? 
candidate  OMAC  was  revised  to  include  the  purview  constraint  as  a 
parameter  and  to  use  the  simpler  OMAC  function  presented  previous. y and 
repeated  here: 

C2  = A x T + B x CPA/TCPA  (for  all  contacts  within 

purview) 
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C . _ = 11  (two  tailed) 


Number  of  Entries  in  Critical  Cells  (1,  2,  3,  7,  8,  9) 


N = 12 

C = 14  (two  tailed) 

• 05 


T+  = 29 


TABLE  6 OLD  vs.  RVV 
Wilcoxon  Test 
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In  order  to  understand  why  a purview  factor  must  be 
included  as  part  of  OMAC,  consider  the  two  paths  shown  in  Figure  8. 

Path  A includes  nodes  a,  d,  c,  e,  f,  and  Path  B includes  nodes  a,  b, 
c,  e,  and  f.  If  own  ship  moves  along  segment  a - d of  Path  A,  it 
approaches  the  closest  point  of  approach  to  the  contact.  Since  the 
OMAC  function  ( C ),  given  above,  includes  a penalty  for  projected 
CPA's  less  than  2 miles,  there  is  a non-zero  penalty  value  produced 
by  the  second  term  of  the  equation.  In  comparison,  segment  b - c of 
Path  B does  not  have  a projected  CPA  violation.  Therefore,  the 
second  term  in  the  equation  is  zero  along  that  segment.  Transit  time 
can  be  the  same  over  segments  a d c and  a b c and  therefore  the 
penalty  due  to  the  time  used  (the  value  of  the  first  term  in  the  equation) 
is  the  same  along  both  Path  segments.  As  a result.  Path  B is  always 
selected  by  the  optimal  synthesis  process. 

This  result  holds  independent  of  the  form  of  the  equation 
and  parameter  values  (A,B)  provided  there  is  a penalty  for  a projected 
CPA  violation  at  some  extended  time. 

But,  subjects  did  not  command  that  type  of  ship  response. 
Instead,  the  ship  is  commanded  along  Path  A toward  point  d until  the 
subject  decided  that  closer  approach  to  the  CPA  was  undesirable.  When 
that  point  was  reached,  a maneuver  to  another  course  such  as  segment  b-c 
was  commanded.  The  purview  limitation  was  included  to  represent  that 
ship  control  characteristic . 
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Analysis  of  results  with  the  one-contact  problem 


established  a set  of  parameter  values  for  test  in  the  25-contact  problem. 
Values  of  purview  selected  were  6.0,  8.3,  10,  12.5  miles.  Values 
of  the  weighting  on  time  (A)  were  selected  as  0.1,  0.25,  and  0.4. 

Recall  that  values  of  B are  obtained  from  A according  to  the  equation: 

B = 1 . - A 

3.4  Analysis  IV  Results:  Comparison  of  Optimal  Ship  Responses 

With  Subject  Controlled  Ship  Responses 

3.4.1  Subject  Controlled  Ship  Response 

A transition  matrix  was  developed  to  represent  the  composite 

performance  of  all  subjects  using  a particular  display.  Three  matrices 

corresponding  to  the  three  display  types  are  given  in  Figures  9,  10  and 

11.  Matrix  entries  are  the  probability  of  transferring  from  Cell  i to 

Cell  j given  a transition  from  Cell  i occurs. 

The  mean  number  of  times  (MT)  a cell  is  entered  in  the 

critical  region  (i.e..  Cells  1-5,  7-11)  before  entering  the  absorbing 

Cell  (21),  given  a starting  entry  in  Cell  5,  are  documented  in  Figure  12. 

Cell  5 was  used  as  a standard  starting  cell  because  it  represents  a 

condition  where  there  is  a projected  CPA  violation  in  20  to  25  minutes, 

and,  thus,  the  ship  control  response  from  that  cell  (as  measured  by  MT) 

reveals  a characteristic  collision  avoidance  response.  This  situation 

was  presented  to  almost  all  subjects  and  therefore  a reasonable  amount 

of  data  is  available. 
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FIGURE  9 CELL  TRANSITION  MATRIX  FOR  PACS  DiSPLAY 


CELL 


FIGURE  10  CELL  TRANSITION  MATRIX  FOR  OLD  DISPLAY 


FIGURE  11  CELL  TRANSITION  MATRIX  FOR  RW  DISPLAY 


Display  Type 


Cell  PACS  OLD  RW 


1 

.03 

.07 

.05 

2 

CO 

o 

.23 

.13 

3 

.20 

.40 

.26 

4 

.45 

.74 

.68 

5 

1.17 

1 .07 

1 .10 

7 

.04 

.14 

.17 

8 

o 

CO 

.14 

.21 

9 

.06 

.17 

.21 

10 

.1  1 

.34 

.33 

1 1 

.23 

. 1 1 

.17 

FIGURE  12  MEAN  NUMBER  OF  TIMES  A CELL  IS  ENTERED 

y 

i 

3-12 




3.4.2 


Optimal  Ship  Response 


Six  sets  of  OMAC  parameter  values  were  used  to  determine 
optimal  ship  response  patterns  for  the  25-contact  problem.  A tabulation 
of  these  parameter  value  sets  is  given  in  Figure  13.  Results  of  the 
tests  are  the  MT  for  each  cell  for  each  OMAC  parameter  set  tested. 
Results  are  given  in  Figure  14. 

3.4.3  Ship  Response  Comparison 

Subject  controlled  ship  response,  as  measured  by  the  MT 
for  each  display  type,  was  compared  to  the  ship  response  for  each 
OMAC.  The  sum  of  squared  differences  of  MT  on  a cell-by-cell 
basis  was  used  for  the  comparison.  Results  of  the  comparison  for 
display  type  vs.  OMAC  purview  are  given  in  Figure  15.  Results  of 
the  comparison  of  display  type  against  the  OMAC  weighting  on  transit 
time  are  given  in  Figure  16.  Best  comparisons  (least  sum  of  squared 
differences)  for  the  PACS,  OLD,  RW  displays  are  12.5,  10,  12.5  miles 
purview  respectively.  Best  comparison  for  time  weighting  is  A = 0.1 
for  all  display  types.  The  percentage  of  sum  of  squares  explained 
for  PACS, OLD,  RW  is  93.1,  88.8,  and  84.3  respectively. 
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Critical  1 

OMAC 

1 

Cell  . 

i 

1 

2 

3 

I 

4 

5 

6 

1 

0.0 

i 

0.0 

1 

| 

0.0 

i 

0.0 

1 

; 0.13 

i 

0.08 

l 

1 

2 

0.0 

0.0 

0.0 

1 

1 

0.26 

I 

1 

0.13 

1 

; 0.51 

3 

0.0 

0.26 

0.69 

| 

1 .05 

0.66 

1 .03 

4 

0.51 

0.81 

1 .03 

1 .02 

0.60 

1 .0 

5 

1 .03 

1 .03 

1 .03 

1 .02 

1 .01 

: i .0 

7 

0.0 

0.0 

0.0 

0.09 

0.01 

0.0 

8 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

9 

0.0 

0.0 

0.03 

0.26 

0.0 

l 

0.0 

- - — 

10 

0.0 

0.0 

0.03 

0.0 

0.0 

o 

• 

o 

1 1 

0.33 

0.0 

0.0 

0.0 

0.0 

0.0 

Entries  are  the  mean  number  of  times  (MT) 
each  cell  is  entered  before  the  system 
enters  the  absorbing  cell  (Cell  21)  given 
the  cell  sequence  started  in  Cell  5. 

FIGURE  14  MEAN  NUMBER  OF  ENTRIES  IN  CRITICAL  CELLS 
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4.0  CONCLUSIONS 

Analysis  I shows  that  performance  differences  between 
certain  display  types  are  significant  at  the  Q = 0.10  level.  Summary 
measures  which  reveal  this  performance  difference  are  time-to-CPA  and 
distance  travelled.  The  remaining  summary  measures  do  not  show 
significant  differences.  Analysis  II  confirms  that  time-to-CPA  is  an 
important  measure  since  with  it  significant  differences  in  performance 
can  be  shown.  Analysis  II  also  reveals,  via  the  measure  MT  (the 
mean  number  of  entries  in  critical  cells),  the  nature  of  the  performance 
differences.  From  these  results  it  is  concluded  not  only  that  there  are 
significant  differences  in  performance  with  different  displays,  but  also 
that  performance  measures  which  detect  responses  leading  to  a critical 
condition  are  more  sensitive  than  summary  measures  of  the  critical 
conditions  themselves. 

Analysis  III  reveals  that  models  of  human  operators  con- 
trolling ships  must  include  a purview  factor.  The  purview  factor  used 
here  was  a radius  within  which  a contact  is  considered.  The  model 
ignores  contacts  outside  that  radius.  Note  that  other  purview  functions 
may  be  useful  but  some  purview  function  was  necessary  to  represent 
subject  performance  as  a function  of  range  to  contact.  The  purview 
function  is  required  to  represent  all  performances,  i.e.,  without 
regard  to  display  type.  It  may  also  explain  differences  in  performance 
with  different  displays;  but,  this  property  was  not  investigated. 
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Analysis  IV  reveals  the  effect  of  OMAC  parameter  values 
on  the  comparison  of  optimal  control  to  the  subject  control  data  obtained 
in  the  experiment.  Results  show  that  performance  is  at  least  a function 


of  transit  time  and  CPA/TCPA  weighting  as  well  as  purview.  Results 
suggests  that  for  the  contact  density  of  the  experiment  problem,  PACS 
and  RW  can  be  rated  as  at  least  12.5  mile  displays  and  that  OLD  is  a 
10  mile  display.  Results  also  suggest  that  a set  of  OMAC  with  multiple 
parameter  changes  - instead  of  one-at-a-time  as  was  the  case  here  - 
may  improve  modeling  accuracy. 

Consider  now  the  primary  purpose  of  the  research  as 
identified  by  the  four  questions  presented  in  the  introduction  which  con- 
cern methods  of  modeling  the  human  operator.  A method  of  computing 
the  apparent  OMAC  from  a set  of  candidate  OMAC,  has  been  demonstrated. 
The  apparent  OMAC  has  the  parameter  values  of  those  analyzed  that 
provide  the  best  comparison  to  the  experimental  data.  Also  demonstrated 
is  a method  for  using  the  OMAC  to  predict  operator  control  actions  in 
specific  problem  situations.  The  method  uses  an  optimal  control  synthesis 
employing  the  representative  OMAC  as  the  criteria. 

It  is  important  to  know  whether  or  not  a display  con- 


figuration can  affect  the  apparent  OMAC . The  results  described  here 
show  that  performance  with  different  displays  is  significantly  different 


and  also,  the  results  support  the  hypothesis  that  the  apparent  OMAC 
changes  with  display  design.  While  the  statistical  significance  of  the  latter 
result  has  not  been  investigated  because  model  validation  is  not 
complete,  the  results  suggest  that  a display  design  affects  the  performance 
criteria  used  by  the  subject.  A contrasting  alternative  is  that  the 
apparent  OMAC  is  constant  with,  variation  in  display  design  (for  example, 
that  OMAC  is  only  a function  of  operator  experience  and  problem 

f 

instructions).  With  the  alternative,  differences  in  performance  may 
exist  with  differences  in  display  but  the  apparent  OMAC  is  not  different. 

Many  important  questions  have  not  been  addressed  by  the 
work  reported  here.  These  include  an  investigation  of  more  complex 
OMAC  functions  incorporating  additional  parameters,  validation  of  the 
operators  models  (OMAC)  and  development  of  methods  of  analyzing  the 
apparent  OMAC  to  evaluate  alternative  designs. 
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